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Abstract 

Students with deficits in social skills have been found to experience both short- and long-term 
problems, including interpersonal conflicts and academic difficulties. These problems are 
compounded for students with emotional and behavioral disorders (EBD). Class-wide function- 
related intervention teams (CW-FIT), a multi-tiered classroom management program, has been 
shown to be effective in increasing on-task behavior and decreasing disruptive behavior of 
students at risk for EBD. The present study examined the effects of CW-FIT on teachers’ ratings 
of students’ social skills, problem behaviors, and academic competence. A randomized control 
trial was completed with 160 elementary school teachers located in 19 schools across three 
states. Teachers completed rating scales on 350 students identified as at risk for EBD, for whom 
consent had been obtained. After being randomly assigned to treatment or control conditions, 
CW-FIT was implemented for approximately four months in treatment classrooms, after which 
teachers completed post-test ratings on all students. CW-FIT implementation was associated with 
significantly improved teacher ratings of social skills and academic competence for students at 
risk for EBD, but no significant changes in teacher ratings of student problem behaviors were 
found. Higher fidelity of CW-FIT implementation was associated with improved outcomes. 
Implications, limitations, and areas for future research are addressed. 

Keywords: elementary schools, emotional disturbances, behavior disorders, prevention, 


positive behavior intervention and supports 
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Effects of CW-FIT on Teachers’ Ratings of Elementary School Students at Risk 
for Emotional and Behavioral Disorders 

The importance of improving school-based treatment for students with or at risk for 
emotional and behavioral disorders (EBD) has been widely noted (Severson, Walker, Hope- 
Doolittle, Kratochwill, & Gresham, 2007; Wagner, Kutash, Duchnowski, Epstein, & Sumi, 
2005). Although 10-20% of school-age children experience significant emotional and behavioral 
problems (Mash & Dozois, 2002), less than 1% have been identified with EBD (Wagner et al., 
2005). Unfortunately, such students are often not identified until they have exhibited serious 
problems over multiple school years (Kauffman, 2005; Reid, Gonzalez, Nordness, Trout, & 
Epstein, 2004; Wagner et al., 2006). Early evidence-based interventions are needed to prevent 
students who are at risk for EBD from experiencing the difficulties typical of students with this 
disorder (Conroy, Hendrickson, & Hester, 2004; Wagner et al., 2005). 
Social Skill Deficits, Problem Behaviors, and Academic Difficulties 

Students classified with EBD consistently demonstrate lower social skills than typical 
peers or their peers with other disabilities (Lane, Carter, Pierson, & Glaser, 2006; Wagner et al., 
2005). Social skills are learned, situation-specific behaviors that result in positive interpersonal 
interactions (Merrell & Gimpel, 1998; Young, Caldarella, Richardson, & Young, 2012). Shores 
and Wehby (1999) noted that the social interactions between teachers and students with EBD 
often consist of more negative than positive or neutral interactions. Indeed, one of the most 
prevalent characteristics of EBD is an inability to build or maintain satisfactory interpersonal 
relationships with others (Gage, 2013; Individuals with Disabilities Education Improvement Act, 
2004). This is concerning since students need to learn and use appropriate social skills to be 


successful in school with teachers, staff, and peers. Hemmeter, Ostrosky, and Fox (2006) noted 
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that social skills are a critical foundation upon which school success is built. They reported a 
significant negative correlation between students’ social skills and their challenging behavior: As 
social skills increased, problem behaviors decreased. These researchers also noted that many 
challenging behaviors are exhibited due to a lack of experience using appropriate social skills. 

Students with EBD have higher levels of problem behaviors than students identified with 
learning disabilities, including significantly higher rates of school absenteeism, suspension, and 
expulsion (Lane et al., 2006; Wagner et al., 2005). Such students tend to misinterpret neutral 
cues as hostile and use poor problem-solving skills, resulting in antisocial behaviors that make it 
difficult to develop adaptive relationships with peers and teachers (Gresham, 2002; Lane, 
Wehby, & Barton-Arwood, 2005). School-based programs designed to improve social 
interactions and teach specific social skills to reduce such problem behaviors have been shown to 
be effective (Cook et al., 2008). 

Students with EBD also struggle academically, earning lower grades, failing more 
classes, repeating a grade, and dropping out of school more often than other students (Wagner et 
al., 2005). In a meta-analysis on the academic status of students with EBD, Reid and colleagues 
(2004) found that such students demonstrate significantly lower academic skills across subjects 
when compared to a norm group without disabilities. Further, students with EBD are often 
viewed as less academically competent than students with learning disabilities (Lane et al., 
2006). Those with EBD may struggle due to academic skill deficits, lack of content knowledge, 
and limited task completion (Lane et al., 2005). 

Positive Behavior Interventions and Support 

Teachers can incorporate effective behavior management strategies for students with, or 

at-risk for, EBD by using positive behavior intervention and supports (PBIS) in their classrooms. 


PBIS is an evidence-based, multi-tiered framework for preventing or eliminating challenging 
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student behaviors using proactive strategies including direct teaching and reinforcing of 
appropriate social skills (Sugai et al., 2000). The guiding principles of PBIS recommend 
application in natural school settings, support for all students, prevention of behavioral problems, 
consistent improvement based on collected data, and systems level change (Carr et al., 2002; 
Sugai et al., 2000). PBIS core components include (a) fostering students’ positive relationships 
and interactions with peers and adults, (b) defining and teaching clear behavioral expectations 
during instruction time, and (c) providing students feedback on their use of appropriate social 
skills throughout the school day (Benedict, Horner, & Squires, 2007; Reinke, Herman, & 
Stormont, 2013; Sugai & Horner, 2009). 

Teachers and school staff can provide effective instruction in both social skills and 
academics using evidence-based behavior management techniques consistently and skillfully 
(Kauffman, 1999). Shores et al. (1993) called for programs that emphasize proactive practices to 
create positive interactions involving students with EBD, their peers, and their teachers. To 
support the success of such students, Benner, Kutash, Nelson, and Fisher (2013) recommended 
using a variety of PBIS strategies including explicit instruction and interdependent group 
contingencies with clear classroom expectations. These effective classroom management 
practices (i.e., classroom rules, increased teacher praise, student feedback, and positive 
reinforcement) provide a foundation for improving classroom engagement and student behavior 
(Conroy, Sutherland, Haydon, Stormont, & Harmon, 2008). Classroom management strategies 
that employ social skills instruction, group contingencies, classroom organization, rules, and 
other similar PBIS strategies need to be further investigated to discover the effects on the social 
skills of students with or at risk for EBD. 


Class-Wide Function-Related Intervention Teams 
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Class-wide function-related intervention teams (CW-FIT; Wills et al., 2010) is a multi- 
tiered classroom management program to aid teachers in consistent and skillful implementation 
of evidence-based strategies. Specifically, it incorporates a variety of PBIS strategies 
implemented at tier 1 (social skills instruction, interdependent group contingencies, and praise) 
and tier 2 (self-management and help cards) to improve classroom management and student 
behavior during typical academic instruction. A number of research studies have been completed 
on the CW-FIT program in elementary schools demonstrating positive results. The program has 
been shown to be effective in increasing on-task behavior at a class level (Caldarella, Williams, 
Hansen, & Wills, 2015; Kamps et al., 2011; Kamps, Wills et al., 2015; Wills et al., 2010) as well 
as decreasing disruptive behavior of students at risk for EBD (Kamps, Conklin, & Wills, 2015; 
Wills, Iwaszuk, Kamps, & Shumate, 2014; Wills, Kamps, Fleming, & Hansen, 2016) when 
implemented in general education classrooms. These studies have also reported improvements in 
teacher praise rates and decreases in teacher reprimands. Favorable teacher and student ratings of 
social validity have also been reported. However, there has yet to be a study demonstrating the 
effects of the program on teachers’ ratings of students’ social skills. Such a study is worthwhile, 
given the importance of social skills in student success (Merrell & Gimpel, 1998). In addition, 
these past studies have focused on direct observations of discrete behaviors displayed during 
CW-FIT implementation, rather than teacher ratings reflecting student behaviors exhibited 
throughout the day. Adding such ratings can help provide a more comprehensive assessment of 
the effects of interventions (Whitcomb & Merrell, 2013). 
Summary and Research Purpose 

Students who lack appropriate social skills have been found to experience both short- and 
long-term problems, including interpersonal conflicts and academic difficulties. These problems 


are compounded for students with EBD. Students with or at risk for EBD need to learn and use 
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appropriate social skills if they are to be successful in school. CW-FIT incorporates strategies 
including social skills instruction to promote students’ appropriate classroom behavior. Although 
prior studies have shown CW-FIT to be effective in increasing on-task behavior and decreasing 
disruptive behavior of students at risk for EBD, additional research is needed to examine the 
effects of CW-FIT on teacher ratings of social skills, problem behaviors, and academic 
competence of these students. Given the results of prior studies, we had four hypotheses in the 
current study: CW-FIT would result in improved teacher ratings of (1) social skills, (2) problem 
behavior, and (3) academic competence for students identified as at-risk for EBD, and that (4) 
higher fidelity of implementation would result in improved outcomes. 
Method 

Participants and Settings 

Demographic data are presented in Table 1. Teacher participants included 149 general 
and 11 special education teachers (n = 160; 95% female, 5% male; 83% Caucasian, 11% 
African-American, 3% Hispanic) from all elementary grades (pre-kindergarten through six). 
Most teachers held bachelor's (45%) or master's (41%) degrees, and the group had an average of 
9 years teaching experience (range = 0 to 44 years). Teachers completed rating scales on 350 
students identified as at risk for EBD (27% female, 73% male; 43% Caucasian, 39% African- 
American, 15% Hispanic). Students were distributed across grade levels: 1% pre-kindergarten, 
18% kindergarten, 18% first, 16% second, 20% third, 13% fourth, 11% fifth, and 4% sixth grade. 
School records were used to obtain student demographic data. Participants were drawn from 19 
schools, mostly Title I, located in urban settings in Missouri, Utah, and Tennessee. All teachers 
and students were participants in a multi-site, multi-year efficacy study of CW-FIT funded by the 
Institute of Education Sciences, with the exception of the pre-kindergarten participants (see 


Jolstead et al., 2016). Teachers and students each participated for one academic year. 
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Measures 

Systematic Screening for Behavior Disorders (SSBD): Stage 1. The SSBD (Walker & 
Severson, 1992) is a nationally normed multi-stage process for identifying elementary students at 
risk for behavior disorders. In Stage 1, teachers nominate and rank order their students who 
exhibit externalizing or internalizing behaviors. Stage 1 inter-rater agreement (Spearman’s rho) 
ranges between .82 and .94, and test-retest reliability ranges between .72 and .79. 

Social Skills Improvement System-Teacher Form (SSIS). The standardized norm- 
referenced SSIS (Gresham & Elliott, 2008) consists of three scales: Social Skills, Problem 
Behaviors, and Academic Competence. There are 76 items rated on a 4-point Likert scale from 
never to almost always and 7 items rated on a 5-point Likert scale from lowest 10% to highest 
10%. Sample items include “Follows your directions,” “Disobeys rules or requests,” and “In 
reading, how does this student compare with other students?” Internal consistencies (alphas) on 
the SSIS scales range from .94 to .97. 

Direct observations. Direct observations of students were conducted by trained research 
staff using the Multi-Option Observation System for Experimental Studies (MOOSES; Tapp, 
Wehby, & Ellis, 1995), a computer software program that allows researchers to record frequency 
and duration events for later analysis. During a 15-min observation of an individual student, 
observers recorded frequency of disruptive behaviors and duration of engagement. Disruptive 
behavior was defined as deliberate verbal, physical, or motor displays of inappropriate behavior 
interfering with a student’s participation and/or the productive classroom activity of peers. 


Engagement was defined as a student appropriately working on the assigned/approved activity. 
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MOOSES has been successfully used in other research studies involving observations of student 
behavior (see e.g., Kamps et al., 2011; Reinke et al., 2013). 

School Social Behavior Scales-Second Edition (SSBS-2). The standardized, norm- 
referenced SSBS-2 (Merrell, 2002) consists of two teacher-rating scales: Social Competence and 
Antisocial Behavior. There are 64 items rated on a 5-point Likert scale from | (never) to 5 
(frequently). Sample items include “Offers to help other students when needed” and “Insults 
peers.” Internal consistencies (alphas) on the SSBS-2 range from .96 to .98. 

Classroom Performance Survey — Elementary (CPS-E). The CPS-E was adapted from 
the Classroom Performance Survey (CPS; Robin, 1998), as explained by Caldarella and 
colleagues (2016). The original CPS was developed due to a need within secondary schools for a 
brief, reliable, behavior rating scale that was effective in identifying school functioning levels for 
students with attention deficit hyperactivity disorder (Robin, 1998). The original CPS consisted 
of 20 questions on a 5-point Likert scale pertaining to a student’s academic and social strengths 
and weaknesses. Brady, Evans, Berlin, Bunford, and Kern (2012) conducted a psychometric 
analysis of the CPS across 23 high schools, 875 students, and 146 teachers. They concluded that 
(a) the CPS was comprised of two factors, Academic Competence and Interpersonal 
Competence; (b) CPS scores could be interpreted similarly across genders; and (c) the CPS could 
serve as a progress monitoring tool for at-risk students. 

We used a modified CPS elementary version, CPS-E (see Caldarella et al., 2016), 
consisting of 17 items across two scales: Academic Competence and Interpersonal Competence. 
All items are rated on a 5-point Likert scale from 1 (always) to 5 (never). Sample items include 
“Completes class assignments,” and “Relates positively to peers.” Internal consistencies (alphas) 
on the CPS-E range from .79 to .92 (Caldarella et al., 2016). Lower scores on both the Academic 


Competence and Interpersonal Competence scales indicate improved behavior. 
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Treatment fidelity. Observers completed an 18-item fidelity form during each 
observation period (approximately 15 total observations per classroom, in both treatment and 
control classrooms). They observed implementation of CW-FIT and scored each component 
(e.g., social skills posted, praise paired with points, teams immediately rewarded) yes or no, if it 
was implemented. The observers recorded yes if a teacher demonstrated any of these components 
in their classroom practice. Before utilizing CW-FIT, treatment teachers implemented 3.26% (SD 
= 3.93) of CW-FIT components in the natural course of their teaching. During the study, 
treatment teachers implemented CW-FIT with 92.08% fidelity (SD = 6.72). In control 
classrooms, 2.43% (SD = 3.84) of components were implemented during baseline and 2.72% of 
CW-FIT components were observed (SD = 3.74) during the course of the study. 

Social validity. Teachers and students in treatment classrooms indicated their level of 
satisfaction with CW-FIT by completing a social validity questionnaire at the end of their 
participating year. The teacher questionnaire included 18 items [15 items rated from | (very true) 
to 4 (not true) and 3 open-ended questions] regarding ease of implementation, acceptability of 
CW-FIT components (use of timer, teams, points, etc.) and perceptions of effectiveness on 
student engagement and classroom behavior. The student questionnaire consisted of two 
questions rated as yes or no (“Did you like playing the CW-FIT game?” and “Do you think other 
kids should get to play the CW-FIT game?”’) and three open-ended questions assessing what 
aspects of CW-FIT they liked or disliked and why. 

Procedures 

Teacher and student identification. Districts referred schools to researchers as potential 
participants. Researchers approached school principals to inquire whether their teachers could 
benefit from a classroom management intervention. A recruitment meeting was held enabling 


teachers to choose to participate. Teachers who volunteered completed necessary informed 
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consent procedures as required by school districts and institutional review boards at the 
participating universities. 

Participating teachers identified the time of day with the greatest number of classroom 
behavioral challenges, during which all data collection took place. Teachers nominated students 
who exhibited externalizing or internalizing behaviors on the SSBD. Parents of the top three 
nominated students (externalizing or internalizing) were contacted and informed parent 
consent/student assent obtained. To be considered at risk and qualify for participation, students 
had to (1) score in the above average range for problem behaviors on the SSIS and (2) be verified 
as at risk via direct observations using MOOSES, as done in past studies of CW-FIT (see e.g., 
Caldarella et al., 2015; Kamps et al., 2011). Students whose MOOSES engagement levels were 
below 75% or whose disruptive behaviors were above 10 occurrences for a minimum of two out 
of five 15-min observation sessions were considered at risk, as similar characteristics have been 
found in other studies of behaviorally at risk students (see e.g., Kamps, Conklin et al., 2015; 
Wills et al., 2014; Wills et al., 2016). 

Assignment to experimental conditions. Teachers were randomly assigned to either 
treatment or control classrooms, after which training occurred and CW-FIT commenced in 
treatment classrooms. Random assignment occurred at the teacher/class level due to the teachers’ 
implementation of the intervention. Teachers’ names were stratified into grade (K-2, 3-6) and 
type of classroom (general education, special education) and randomly assigned by the 
researchers using the randomized selection function in Excel. The decision as to when to begin 
CW-FIT implementation in assigned classrooms was based on the stability of students’ 
engagement behavior. Safeguards to prevent contamination included (1) direct observations in 
control classrooms using the same treatment fidelity form used in treatment classrooms, (2) 


agreements from each teacher at the recruitment meeting that those randomly drawn as control 
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classes would not use the intervention during the study, and (3) delayed CW-FIT training for 
control classes until after completion of the study. 

Treatment classrooms. Teachers received training after baseline data were collected and 
just before CW-FIT was implemented. Research staff conducted a two-hour training session 
during which the rationale and logistics of CW-FIT were explained and opportunities to practice 
components were provided. Teachers were given scripted lessons to introduce social skills and 
were provided feedback as they practiced. Trainers emphasized the value of using praise and 
included videos of other teachers implementing CW-FIT in their classrooms. To help embed the 
intervention into the classroom, teachers were instructed to use it as part of their regular 
academic instruction, where they taught as usual, and to supplement with CW-FIT to manage 
behavior. Trainers coached teachers (i.e., answering questions, providing feedback on quality of 
CW-FIT implementation) for one to two weeks after the training until teachers were able to 
independently implement with fidelity as indicated by fidelity observations. Coaching length 
varied based on how quickly teachers were able to implement CW-FIT independently. 
Intervention data were collected after training and coaching were completed. 

Teachers directly taught CW-FIT social skills to students during 10-min lessons by 
defining the skills, facilitating discussions regarding rationales for the skills, and role-playing 
with students with follow-up discussions. The three main social skills were “Follow Directions 
the First Time,” “How to Get the Teacher’s Attention,” and “Ignore Inappropriate Behavior.” 
Each skill included four specific steps with visual cues posted in a visible location in the 
classroom. Teachers could choose additional social skills to teach as needed, including “Talk in a 
Quiet Voice,” “Keep Hands, Feet, and Objects to Self,” or “Stay in Your Seat.” After all social 


skills were taught (one skill per day), teachers briefly reviewed the skills with students each day 
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before starting academic lessons in which CW-FIT was used. Throughout the academic lesson, 
teachers referred to the social skills. 

Additionally, teachers stated daily point goals to guide the students and encourage 
demonstration of social skills. These goals were set at approximately 75-85% of the number of 
time intervals that would occur during the lesson. Teachers set a timer to sound at approximately 
3-min intervals, though this was adjusted based on behavioral needs in classrooms (e.g., younger 
students needed shorter intervals; intervals could be increased if students were consistently on 
task). Teachers divided their classes into teams of two to six students, based on classroom 
conditions and convenience, to establish a group contingency. When the timer sounded, teachers 
scanned each team. If every student on a team was displaying appropriate social skills and was 
on task at that moment, teachers praised the team and marked a point on a displayed point chart 
(points earned were never taken away for misbehavior). The goal of working in teams was for 
students to provide positive peer influence and encourage appropriate behavior. At the 
conclusion of the CW-FIT session, teams that met the predetermined point goal received an 
immediate reward. The reward was short and simple, feasible for teachers, and reinforcing to 
students (e.g., stickers, positive notes, simple games, dance time). Through direct instruction, 
goals, points, teams, and rewards (e.g., group contingency), teachers attended to appropriate 
behaviors frequently and taught ways for students to gain help or attention in appropriate ways, 
as well as ignore inappropriate behaviors. These strategies, in combination with self- 
management and help cards described below, address two commonly reported functions of 
behavior, attention (from teacher or peers) and escape (from academic tasks; Ervin et al., 2001). 

For students who needed additional supports, two options were available: self- 
management charts and help cards. Self-management charts, on which students self-record points 


when the timer sounds if they are on task displaying appropriate social skills, were used to help 
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students who displayed attention-seeking behaviors. Help cards, which students use to solicit 
academic assistance from teachers or peers, were used for students engaging in escape or 
avoidance behaviors due to academic struggles. 

Control classrooms. Control classroom teachers taught using a “business as usual” 
approach, implementing their typical classroom management strategies. Examples of the typical 
strategies used by these teachers included praise, reprimands, behavior charts, clip charts, color 
coded behavioral cards, token economies, PBIS tickets, ClassDojo (see www.classdojo.com), 
daily behavior report cards, and class rewards. As noted earlier, fidelity observations revealed 
that control classroom teachers implemented 2.72% of CW-FIT strategies throughout the study. 

Data collection schedule. During an approximately three week baseline phase (before 
CW-FIT), teachers completed rating scales on each student identified as at risk. One SSIS and 
SSBS-2 were administered, as these measures are designed to assess student behavior displayed 
over a longer period of time. Items directly related to the three main social skills taught during 
CW-FIT (“Follow Directions the First Time,” “How to Get the Teacher’s Attention,” and 
“Ignore Inappropriate Behavior”) are included on both the SSIS and SSBS-2 rating scales. Three 
CPS-E, one per week, were administered during baseline, as this measure is designed to be used 
as a behavioral progress monitoring tool (Caldarella et al., 2016). During intervention, teachers 
completed one CPS-E per month. Approximately four months after baseline, at the end of CW- 
FIT implementation, teachers again completed rating scales on participating students (though 
only one final CPS-E), as well as the social validity questionnaire. All students in participating 
treatment classrooms also completed a social validity questionnaire. This process was replicated 
across all locations, for all three years of the study. 


Design and Analysis 
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A causal randomized controlled trial (RCT; Moher, Schulz, & Altman, 2001) was 
conducted to address the research questions. The design allowed for a comparison of the effects 

of a treatment group using CW-FIT versus an equivalent control group using only the teachers’ 
prior methods of classroom management (Shadish, Cook, & Campbell, 2002). To establish 
baseline equivalence of groups on the pre-test measures, a Robust Means Modeling (RMM) 
analysis was completed. The RMM analysis is similar to ANOVA in that it allows comparisons 
of means, but has an advantage over traditional ANOVA because it does not require normality or 
equality of variances (Fan & Hancock, 2012). The RMM analysis and all subsequent analyses 
were conducted using Mplus 7.4 (Muthén & Muthén, 1998-2015). 

Several analyses were then conducted to answer the research questions: (a) a RMM 
analysis to test whether CW-FIT results in improved scores on the SSBS-2 and SSIS measures 
for students at risk for EBD; (b) a latent growth curve model studying the progress of students 
across all eight time points of the CPS-E measure; and (c) a multivariate linear regression 
analysis, simultaneously regressing fidelity of treatment on all outcomes of interest. Different 
statistical techniques were used to answer different questions: The RMM answered causal 
questions generated from the RCT, while the fidelity analysis was exploratory and should not be 
considered to have the same causal impact. 

The assumptions for all analyses included independence of observations, which was 
violated in the present study because students were nested within classrooms. This fact was 
handled in MPLUS 7.4 by using the TYPE=COMPLEX option for the RMM and fidelity 
analysis and TYPE=TWOLEVEL for the latent growth curve model with teacher ID being the 
cluster variable. Missing data were considered missing at random and handled by the full 
information maximum likelihood method (FIML) in Mplus, which computes a likelihood 


function using only observed variables. Enders and Bandalos (2001, p. 434) explained that 
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“Although...the FIML algorithm does not impute missing values, this borrowing of information 
from the observed portion of the data is conceptually analogous to replacing missing Y data 
points with the conditional expectation of Y given X.” FIML has been shown to have better 
results than other missing data techniques such as listwise deletion (Enders, 2010). 

The latent growth curve model assumed the two factor structure for the CPS-E measure 
as previously studied (Caldarella et al., 2016), namely Academic Competence and Interpersonal 
Competence. Each was measured across eight time points, and thus each process had its 
respective intercept and slope. The latent intercept for both processes was set at the last time 
point to test the differences of the measures at the end of the study. Treatment was regressed on 
the latent intercept and latent slope for both processes to study the effect of treatment on 
students’ progression by the end of the study and how the rate of change was affected. The 
fidelity analysis consisted of multivariate linear regression of fidelity on outcomes within the 
treatment and control groups separately, due to significant differences between groups. 

Results 
Baseline Equivalence 

Results for the RMM analysis of baseline measures showed that none of the SSIS and 
SSBS measures were significantly different at baseline (p > .05), except for one SSBS subscale 
(Defiant/Disruptive, p = .03) which showed the treatment group was higher (worse) by 1.86 
points. Latent growth curve modeling showed the CPS-E measures were also higher (worse) for 
the treatment group at baseline for both Academic and Interpersonal factors. The fact that the 
treatment group was worse than the control group at baseline on these measures increased 
confidence that selection effects would not explain any improvement at the end of the study. 


Robust Means Modeling 
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Results for the RMM are presented in Table 2. The p values for testing whether the 
treatment positively affected these measures were all one-sided, corresponding to the directional 
hypotheses. Cohen’s d was also reported for all comparisons as an effect size estimate. The 
RMM results showed statistically significant outcomes favoring the treatment group over the 
control group for (a) SSBS-2 Social Competence Total, (b) SSBS-2 Peer Relations, (c) SSIS 
Communication, (d) SSIS Cooperation, (e) SSIS Assertion, (f) SSIS Empathy, and (g) SSIS 
Engagement. The Cohen’s d for these measures ranged from 0.20 to 0.27, which are considered 
small effect sizes. The RMM showed that none of the problem behavior measures had p values 
below or near .05, indicating no treatment effect. The RMM showed academic behavior as 
measured by the SSBS-2 as having statistically significant results, with a p = .004 and a Cohen’s 
d of 0.27 showing a small effect size. There were no significant differences on the SSIS 
Academic Competence scale. 
Latent Growth Curve Model 

Changes in the CPS-E factors over time are shown in Figure 1. It is important to note that 
lower scores are preferable on this measure. All reported p values are one sided, corresponding 
to the directional hypotheses. While there were improvements in both groups over time, the 
statistical analyses suggested greater improvement in the treatment group than the control group. 
The CPS-E Academic Competence score was significantly lower (0.20 points, 0.30 standard 
deviation units) for the treatment group compared to the control group (p = .011) at the last time 
point, indicating that teachers rated students in the treatment group as significantly more 
improved than students in the control group. The effect of treatment on the slope was also 
statistically significant (b = -0.03, p < .001). In the control group Academic Competence scores 
decreased by 0.03 points (0.04 standard deviation units) for every one-unit increase in time on 


average. In the treatment group Academic Competence scores decreased by 0.06 points (0.09 
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standard deviation units) for every one-unit increase in time: Thus CW-FIT approximately 
doubled the natural improvement in this measure. 

There was no significant difference between the treatment and control groups on CPS-E 
Interpersonal Competence scores (p > .05) at the last time point. However, the effect of treatment 
on the slope was statistically significant (b = -0.03, p = .006). In the control group, Interpersonal 
Competence decreased by 0.03 points (0.07 standard deviation units) per one-unit increase in 
time on average. In the treatment group Interpersonal Competence decreased by .06 points (0.14 
standard deviation units) per unit of time: Thus, the effect of CW-FIT approximately doubled the 
natural improvement in this measure also. 

Fidelity Results 

Fidelity results are reported in Table 3. It was necessary to run separate analyses for the 
treatment and control groups as the mean difference in fidelity was high (as earlier mentioned), 
thus violating normality assumptions. All reported p values were one-sided, corresponding to the 
directional hypothesis. A standardized Beta (8), as produced by Mplus, was included as an effect 
size. In the control group, only one relation was significant (SSBS-2 Social Competence total b = 
0.41, p = .038, B = 0.14) and the non-significant results are not reported. The fidelity results 
reported below are for the treatment group only. 

Fidelity results for the treatment group showed that effects for SSBS-2 Social 
Competence Total, SSBS-2 Self-Management/Compliance, and CPS-E Interpersonal 
Competence were all significant (p < .05) suggesting that an increase of fidelity affected these 
social skill measures in a positive way. The absolute value of the standardized fs ranged from 
0.15 to 0.20, which are considered small effect sizes. 

Fidelity results for the treatment group also showed that p values were less than .05 for 


SSBS-2 Antisocial/Aggressive, SSIS Hyperactivity/Inattention, and SSIS Autism Spectrum, 
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suggesting that higher fidelity affected all of these problem behavior measures in a positive way. 
The absolute value of the standardized fs ranged from 0.12 to 0.18, which are considered small 
effect sizes. 

Finally, fidelity results for the treatment group were also positive for SSBS-2 Academic 
Behavior and CPS-E Academic Competence (p < .05), suggesting that higher fidelity affected 
these academic measures in a positive way. The absolute value of the standardized Bs ranged 
from 0.14 to 0.22, which are considered small effect sizes. 

Social Validity 

Results indicated 88% of teachers enjoyed teaching with CW-FIT; 94% found the 
program easy to learn and implement; 96% found teams and points helpful in improving student 
behavior; 92% learned new skills to help manage student behavior; 92% would use CW-FIT 
skills with future classes; 90% would recommend the program to colleagues; 96% reported that 
students liked CW-FIT; and 100% agreed that students were more focused and engaged when 
CW-FIT was implemented. Results indicated 93% of students liked CW-FIT, particularly the 
group rewards and point system; 90% of students agreed that their peers should get to use CW- 
FIT because it is fun and helps improve behavior and teamwork in class. 

Discussion 

The purpose of this study was to investigate the effects of CW-FIT on teachers’ ratings of 
students’ social skills, problem behaviors, and academic competence, as well as the impact of 
treatment fidelity on outcomes. A discussion of the results regarding the four research 
hypotheses follows. Our first hypothesis was that CW-FIT would result in improved teacher 
ratings of social skills for students at risk for EBD. Results showed significant post-test 
improvements in social skills for students in treatment classrooms compared to those in control 


classrooms. These results correspond to prior studies of CW-FIT which have shown 
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improvements in students’ classroom behavior following the intervention (Caldarella et al., 
2015; Kamps et al., 2011; Kamps, Wills, et al., 2015; Wills et al., 2010; Wills et al., 2016). 
Results make sense as CW-FIT involves social skills being directly taught and reinforced by 
classroom teachers. These results are important since improvements in at-risk students’ social 
skills increases the probability of their success in school (Hemmeter et al., 2006) and can result 
in improved classroom learning environments for all students (Conroy et al., 2008). It is 
important to note that significant differences were found for the majority of social skills 
measured, with the exception of skills associated with self-management and self-control. It is 
unclear why no significant differences were found between groups on these variables. 

The second hypothesis was that CW-FIT would result in improved teacher ratings of 
problem behaviors for students at-risk for EBD. However, no significant differences in problem 
behavior ratings were found at post-test. These results are somewhat surprising, since prior 
studies have shown significant decreases in disruptive student behaviors during direct 
observations (Kamps, Conklin et al., 2015; Wills et al., 2014; Wills et al., 2016). However, in 
these prior studies direct observations measured different discrete behaviors than were measured 
in the current study. For example, we measured ratings of behaviors such as blaming others for 
problems, taking things belonging others, and being overly demanding of teacher attention. Prior 
CW-FIT studies measured behaviors such as talking to peers without permission, name-calling, 
and throwing materials (see e.g., Kamps, Conklin et al., 2015). Additionally, when teachers 
completed rating scales, their ratings likely reflected student behaviors exhibited throughout the 
day, not being limited to behaviors only observed while the intervention was being implemented 
(as has been reported in prior CW-FIT studies). 

The third hypothesis was that CW-FIT would result in improved teacher ratings of 


academic competence for students at risk for EBD. Significant improvements in treatment 
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students compared to control students on both the SSBS-2 and CPS-E measures supported this 
hypothesis. These results are important, given the critical role of such skills in students’ 
academic success (Hemmeter et al., 2006). A possible reason the SSIS Academic Competence 
score was not significant may be that it consisted of ranking students compared to other students 
in their classrooms, which may be less sensitive to absolute changes in individual students, 
whereas the SSBS-2 and CPS-E use scales that do not involve such cross student comparisons. 

The final hypothesis was that higher fidelity of CW-FIT implementation would result in 
improved outcomes. Improved fidelity was associated with better outcomes on many of the 
measures in the treatment group, consistent with prior studies showing the importance of CW- 
FIT fidelity on student outcomes (e.g., Caldarella et al., 2015; Kamps, Wills et al., 2015). Results 
also suggest that elementary school teachers can implement CW-FIT with fidelity, which is also 
consistent with these past studies. Additionally, results indicated that these practices, as 
measured by the fidelity observations, were not naturally occurring at high frequencies in control 
classrooms. 
Implications 

There are several implications of these findings. CW-FIT appears to be a feasible and 
socially valid approach to help elementary school teachers implement early, evidence-based, 
PBIS interventions for students at-risk for EBD. The significant increases in teacher ratings of 
social skills and academic competence were impressive, given the moderate length of the 
intervention. This seems a wise investment, considering the negative outcomes for such students 
if not provided treatment (Conroy et al., 2004; Wagner et al., 2005). CW-FIT is administered 
universally ensuring at-risk students can remain in class during academic instruction, thus 
decreasing the likelihood of falling behind academically. Students have the opportunity to 


practice social skills with peers in class rather than with a few select individuals in a pullout 
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group. Since one defining characteristic of EBD is the inability to build or maintain relationships, 
interventions such as CW-FIT that directly teach and reinforce social skills are needed to help 
students improve their interactions with teachers and peers. 
Limitations and Directions for Future Research 

There were several limitations to this study to address in future research. Teacher rating 
scales were the only outcome measures examined. Behavior rating scales have been criticized for 
being more subjective, as they are based on raters’ perceptions of behavior (Cook, Volpe, & 
Delport, 2014). Teachers who implemented the intervention may have been particularly looking 
for positive results (expectancy effects). This is somewhat suggested in the decreasing baseline 
trends in the CPS-E scores in the treatment group (as shown in Figure 1) indicating improvement 
in student ratings. However, behavioral improvements were not consistent across measures (with 
more favorable results seen on measures of social skills and academic competence than on 
problem behaviors), an argument against teachers’ ratings reflecting the effects of subjective 
positive expectancies. Future research in this area would benefit from adding other measures of 
student outcomes including office discipline referrals, systematic direct observations, direct 
behavior ratings, and standardized academic test scores. Although this study spanned three 
states, it was not nationally representative. The study examined only elementary school students. 
The teacher rating scales were not counterbalanced to control for order effects. Finally, the study 
followed students for only one year. Future research would benefit from addressing these 
limitations by including more nationally representative samples, exploring use of CW-FIT in 
secondary schools, counterbalancing measures, and following students over longer periods of 
time to determine whether there are lasting effects. 


Conclusion 
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In conclusion, CW-FIT appears to be a promising intervention for improving outcomes 
for elementary students at risk for EBD. These results are important, given the critical role of 
social skills and academic competencies in student success. Without intervention, such students 
are at increased risk for a variety of negative outcomes including school failure, dropout, and 
later employment difficulties (Lane et al., 2006; Wagner et al., 2005). To support the academic 
engagement of these students, experts recommend using a variety of PBIS strategies including 
classroom rules and expectations, explicit instruction, increased teacher praise, student feedback, 
and positive reinforcement, as well as group contingencies (Benner et al., 2013; Conroy et al., 
2008). Teachers can assist in prevention of and intervention for EBD by providing effective 
instruction in both social skills and academics using evidence-based behavior management 
techniques consistently and skillfully (Kauffman, 1999). The CW-FIT program, which employs 
such strategies in classroom settings, appears to be a viable approach for early intervention in 


elementary schools. 
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Table 1 
Descriptive Data for Teachers (n = 160), Students (n = 350), and Schools (n = 19) 


Name Elementary Numberof Numberof  Percentof Free/reduced School 


of site school teachers students total sample price lunch Size 
Site | School 1 9 18 5.1% 71.9% 220 
School 2 4 12 3.4% 48.2% 243 
School 3 8 17 4.9% 95.8% 319 
School 4 10 20 5.7% 59.2% 578 
School 5 5 10 2.9% 81.2% 290 
School 6 2 17 4.9% 72.7% 289 
School 7 15 36 10.3% 65.0% 515 
Site2 School 8 10 32 9.1% 69.2% 425 
School 9 10 20 5.7% 81.0% 490 
School 10 BS) 12 3.4% 35.9% 476 
School 11 10 14 4.0% 55.3% 409 
School 12 8 18 5.1% 34.0% 630 
School 13 13 29 8.3% 82.7% 504 
Site 3 School 14 8 18 5.1% 52.0% 519 
School 15 7 16 4.6% 94.2% 677 
School 16 9 23 6.6% 98.1% 475 
School 17 5 7 2.0% 40.9% 472 
School 18 8 18 5.2% 91.5% 317 


School 19 7 13 3.7% 99.0% 384 
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Table 2 


Robust Means Modeling Results of the Effect of CW-FIT Treatment on Outcomes of Interest at 


Post-test (Student n=177 treatment, 134 control, 311 total) 


Mean treatment Meancontrol p value of 


vanable post-test post-test differences Soars 
SSBS-2 Social Competence Total 99.30 91.55 .004* 0.35 
Peer Relations 42.90 38.62 .002* 0.39 
Self-Management/Compliance 30.99 29:59 088 0.18 
Academic Behavior 25.15 23.34 .004* 0.27 
SSBS-2 Antisocial Behavior Total 78.95 Lio 379 0.04 
Hostile/Irritable 34.63 34.40 439 0.02 
Antisocial/Aggressive 22.599 21.76 .207 0.10 
Defiant/Disruptive 21.68 21.64 A85 0.00 
SSIS Social Skills Total 71.19 67.33 052 0.19 
Communication 11.86 11.21 .048* 0.20 
Cooperation 8.47 7.62 .006* 0.27 
Assertion 11.84 11.05 .008* 0.23 
Responsibility 8.48 8.31 337 0.05 
Empathy 9:52 8.73 .044* 0.20 
Engagement 11.97 11.07 .030* 0.24 
Self-Control 9.53 922 300 0.07 
SSIS Problem Behaviors Total 28.46 29.68 285 -0.09 
Externalizing 13.68 13.56 A449 0.02 
Bullying 3.58 3.50 A432 0.03 
Hyperactivity/ Inattention 10.15 10.64 214 -0.11 
Internalizing 4.88 5.64 .100 -0.19 
Autism Spectrum 15.79 16.56 .170 -0.12 
SSIS Academic Competence 12.16 11.88 365 0.04 


Note. Mean scores on the SSBS-2 and SSIS are raw scores. Higher scores on the SSBS-2 and 
SSIS indicate higher levels of behavior. *p < 0.05. All p values are one-tailed. 
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Table 3 


Effects of Fidelity on Outcomes of Interest in the Treatment Group (n=177) 


Effect of fidelity on response 


Response variable b SE Pp B 
SSBS-2 Social Competence Total 0.31 0.14 .013* 0.20 
Peer Relations 0.03 0.08 340 0.03 
Self-Management/ Compliance 0.14 0.08 033" 0.15 
Academic Behavior 0.45 0.26 .042* 0.14 
SSBS-2 Antisocial Behavior Total -0.26 0.20 094 -0.15 
Hostile/Irritable -0.14 0.14 ALS7 -0.11 
Antisocial/A ggressive -0.19 0.11 .040* -0.18 
Defiant/Disruptive -0.59 0.44 087 -0.15 
SSIS Social Skills Total 0.24 0.27 187 0.08 
Communication -0.39 0.18 .987 -0.19 
Cooperation -0.01 0.11 .646 -0.01 
Assertion 0.08 0.05 052 0.16 
Responsibility 0.04 0.05 195 0.09 
Empathy 0.00 0.04 A78 0.01 
Engagement 0.04 0.06 242 0.08 
Self-Control 0.00 0.05 A480 0.00 
SSIS Problem Behaviors Total 0.10 0.04 985 0.18 
Externalizing 0.01 0.06 953 0.01 
Bullying -0.15 0.09 O57 -0.14 
Hyperactivity/ Inattention -0.07 0.04 .047* -0.15 
Internalizing -0.08 0.06 099 -0.11 
Autism Spectrum -0.06 0.04 .039* -0.12 
SSIS Academic Competence -0.18 0.09 .974 -0.19 
CPS-E Academic Competence -0.03 0.01 .005* -0.22 
CPS-E Interpersonal Competence -0.02 0.01 .032* -0.15 


Note. Higher scores on the SSBS-2 and the SSIS indicate higher levels of behavior, whereas 
lower scores on the CPS-E indicate improvement. *p < 0.05. All p values are one-tailed. 
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Figure I. Mean scores of CPS-E academic and interpersonal competence factors over time. 
Lower scores on the CPS-E indicate improvement. 
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